Aiming at the transmission performance requirements of Real-Time (RT) services and Non-Real-Time (NRT) services for multi-user in the downlink transmission of Long Term Evolution (LTE) mobile communication system, an improved Modified Largest Weighted Delay First (MLWDF) scheduling algorithm based on weighted average delay was proposed. On the basis of considering both channel perception and Quality of Service (QoS) perception, a weighted average dealy factor reflecting the state of the user buffer was utilized, which was obtained by the average delay balance of the data to be transmitted and the transmitted data in the user buffer. The RT service with large delay and traffic is prioritized, which improves the user performance experience.Theoretical analysis and link simulation show that the proposed algorithm improves the QoS performance of RT services on the basis of ensuring the delay and fairness of each service. The packet loss rate of RT service of the proposed algorithm decreased by 53.2%, and the average throughput of RT traffic increased by 44.7% when the number of users achieved 50 compared with MLWDF algorithm. Although the throughput of NRT services are sacrificed, it is still better than VT-MLWDF (Virtual Token MLWDF) algorithm. The theoretical analysis and simulation results show that transmission performances and QoS are superior to the comparison algorithm.